With the rise of cloud computing, various situations arise that involve accessing large data stores (defined as data stores or data objects that are over 100 megabytes in size) over a communications network. Accessing large amounts of data over a busy network like the Internet, however, is associated with a number of drawbacks. Limited broadband can limit access to the data store, and long access times can render the system unusable. Furthermore, access to commercially available databases often requires a license, which brings up financial considerations when the number of users or computers accessing the database increases.
A well known example that involves accessing large data stores over a communications network involves the airline industry. The Airline Tariff Publishing Company, or ATPCO, is a corporation that publishes the latest airfares for more than 500 airlines multiple times per day. ATPCO provides fare data in an electronic format with the encoded rules associated with those fares, which makes the information suitable for computer processing. Users of the fare data and rules provided by ATPCO include travel agents, computer reservation systems of airlines, and other service providers in the travel industry. As one may imagine, the amount of fare data and rules is very large. When a user of the fare data is first introduced to the system, the user must download from ATPCO an initial data store with a large size—from tens of Gigabytes up to 1 Terabyte in size. The initial data store reflects all fare data and rules for the last 12 months. Subsequently, the new user must download updates of the fare data and rules from ATPCO at least daily—sometimes up to four times daily. Each update can be from 100 megabytes to several gigabytes in size.
All of the fare data and rules that are downloaded must be easily accessible to clients over a communications network. For example, if the user of the data is an airline computer reservation system, then the data store must be accessible to hundreds or thousands of remotely located reservations personnel making reservations for passengers at the same time. Further, for customer service reasons, reservations personnel must be able to access the fare data and rules quickly. Due to the large amount of data being stored and remotely accessed via a communications network, however, storage of the fare data and rules on a standard hard disk drive can result in long access times that render the system unusable. Further, when the data store must be accessible to a large number of clients, then the traffic may interfere with access times as well. In another example, if the user of the data is an online travel agency, the data store must be easily (and quickly) accessible to hundreds and sometimes thousands of clients requesting fare data over the Internet at the same time. If the traffic attributed to data requests over-burdens the system, then access times suffer. Lastly, most commercially available databases require licenses for each entity accessing the database. Thus, many paradigms designate a single server that acts as the requesting node for the licensed database. This arrangement, however, can over-burden the requesting node during busy periods and is not optimal for efficiency reasons.
Therefore, a need exists to overcome the problems with the prior art, and more specifically, there is a need for a more efficient system and method for accessing large data stores over a communications network.